from sklearn_benchmarks.reporting.hp_match import HPMatchReporting

scikit-learn vs. ONNX Runtime

reporting = HPMatchReporting("onnx", config="config.yml")
reporting.run()

KNeighborsClassifier_brute_force

onnx (1.10.1) vs. scikit-learn (1.0.dev0)

All estimators share the following hyperparameters: algorithm=brute.

	estimator	function	n_samples_train	n_samples	n_features	mean_duration	std_duration	iteration_throughput	latency	n_jobs	n_neighbors	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	KNeighborsClassifier_brute_force	predict	100000	1000	100	16.904	0.015	0.0	0.017	-1	1	0.663	1.875	0.022	9.015	0.104	See	See
1	KNeighborsClassifier_brute_force	predict	100000	1000	100	16.904	0.015	0.0	0.017	-1	1	0.663	0.025	0.003	673.120	69.101	See	See
2	KNeighborsClassifier_brute_force	predict	100000	1	100	0.297	0.007	0.0	0.297	-1	1	1.000	1.875	0.022	0.159	0.004	See	See
3	KNeighborsClassifier_brute_force	predict	100000	1	100	0.297	0.007	0.0	0.297	-1	1	1.000	0.025	0.003	11.840	1.249	See	See
4	KNeighborsClassifier_brute_force	predict	100000	1000	100	16.820	0.037	0.0	0.017	-1	5	0.757	2.739	0.033	6.141	0.074	See	See
5	KNeighborsClassifier_brute_force	predict	100000	1000	100	16.820	0.037	0.0	0.017	-1	5	0.757	0.025	0.003	662.641	78.091	See	See
6	KNeighborsClassifier_brute_force	predict	100000	1	100	0.301	0.008	0.0	0.301	-1	5	1.000	2.739	0.033	0.110	0.003	See	See
7	KNeighborsClassifier_brute_force	predict	100000	1	100	0.301	0.008	0.0	0.301	-1	5	1.000	0.025	0.003	11.860	1.432	See	See
8	KNeighborsClassifier_brute_force	predict	100000	1000	100	16.926	0.031	0.0	0.017	1	100	0.882	2.029	0.002	8.344	0.017	See	See
9	KNeighborsClassifier_brute_force	predict	100000	1000	100	16.926	0.031	0.0	0.017	1	100	0.882	0.022	0.003	768.765	88.681	See	See
10	KNeighborsClassifier_brute_force	predict	100000	1	100	0.303	0.006	0.0	0.303	1	100	1.000	2.029	0.002	0.149	0.003	See	See
11	KNeighborsClassifier_brute_force	predict	100000	1	100	0.303	0.006	0.0	0.303	1	100	1.000	0.022	0.003	13.748	1.605	See	See
12	KNeighborsClassifier_brute_force	predict	100000	1000	100	16.874	0.018	0.0	0.017	-1	100	0.882	2.727	0.031	6.188	0.071	See	See
13	KNeighborsClassifier_brute_force	predict	100000	1000	100	16.874	0.018	0.0	0.017	-1	100	0.882	0.024	0.002	700.944	50.155	See	See
14	KNeighborsClassifier_brute_force	predict	100000	1	100	0.315	0.006	0.0	0.315	-1	100	1.000	2.727	0.031	0.115	0.003	See	See
15	KNeighborsClassifier_brute_force	predict	100000	1	100	0.315	0.006	0.0	0.315	-1	100	1.000	0.024	0.002	13.067	0.971	See	See
16	KNeighborsClassifier_brute_force	predict	100000	1000	100	16.806	0.025	0.0	0.017	1	5	0.757	2.024	0.004	8.302	0.020	See	See
17	KNeighborsClassifier_brute_force	predict	100000	1000	100	16.806	0.025	0.0	0.017	1	5	0.757	0.021	0.000	805.331	11.877	See	See
18	KNeighborsClassifier_brute_force	predict	100000	1	100	0.305	0.005	0.0	0.305	1	5	1.000	2.024	0.004	0.151	0.003	See	See
19	KNeighborsClassifier_brute_force	predict	100000	1	100	0.305	0.005	0.0	0.305	1	5	1.000	0.021	0.000	14.624	0.326	See	See
20	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.224	0.061	0.0	0.017	1	1	0.663	1.174	0.008	14.667	0.108	See	See
21	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.224	0.061	0.0	0.017	1	1	0.663	0.020	0.000	868.288	13.271	See	See
22	KNeighborsClassifier_brute_force	predict	100000	1	100	0.303	0.005	0.0	0.303	1	1	1.000	1.174	0.008	0.258	0.005	See	See
23	KNeighborsClassifier_brute_force	predict	100000	1	100	0.303	0.005	0.0	0.303	1	1	1.000	0.020	0.000	15.283	0.340	See	See
24	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.703	0.003	0.0	0.004	-1	1	0.896	1.726	0.012	2.146	0.015	See	See
25	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.703	0.003	0.0	0.004	-1	1	0.896	0.007	0.003	506.484	188.457	See	See
26	KNeighborsClassifier_brute_force	predict	100000	1	2	0.239	0.004	0.0	0.239	-1	1	1.000	1.726	0.012	0.139	0.003	See	See
27	KNeighborsClassifier_brute_force	predict	100000	1	2	0.239	0.004	0.0	0.239	-1	1	1.000	0.007	0.003	32.707	12.184	See	See
28	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.603	0.006	0.0	0.004	-1	5	0.922	2.611	0.016	1.380	0.009	See	See
29	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.603	0.006	0.0	0.004	-1	5	0.922	0.010	0.005	365.663	183.853	See	See
30	KNeighborsClassifier_brute_force	predict	100000	1	2	0.239	0.005	0.0	0.239	-1	5	1.000	2.611	0.016	0.091	0.002	See	See
31	KNeighborsClassifier_brute_force	predict	100000	1	2	0.239	0.005	0.0	0.239	-1	5	1.000	0.010	0.005	24.240	12.197	See	See
32	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.703	0.005	0.0	0.004	1	100	0.929	1.940	0.001	1.909	0.003	See	See
33	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.703	0.005	0.0	0.004	1	100	0.929	0.003	0.000	1321.519	119.458	See	See
34	KNeighborsClassifier_brute_force	predict	100000	1	2	0.240	0.004	0.0	0.240	1	100	1.000	1.940	0.001	0.124	0.002	See	See
35	KNeighborsClassifier_brute_force	predict	100000	1	2	0.240	0.004	0.0	0.240	1	100	1.000	0.003	0.000	85.567	7.854	See	See
36	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.727	0.050	0.0	0.004	-1	100	0.929	2.617	0.013	1.424	0.020	See	See
37	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.727	0.050	0.0	0.004	-1	100	0.929	0.011	0.005	349.586	166.667	See	See
38	KNeighborsClassifier_brute_force	predict	100000	1	2	0.240	0.005	0.0	0.240	-1	100	1.000	2.617	0.013	0.092	0.002	See	See
39	KNeighborsClassifier_brute_force	predict	100000	1	2	0.240	0.005	0.0	0.240	-1	100	1.000	0.011	0.005	22.508	10.736	See	See
40	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.656	0.005	0.0	0.004	1	5	0.922	1.936	0.003	1.888	0.004	See	See
41	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.656	0.005	0.0	0.004	1	5	0.922	0.003	0.000	1316.140	99.460	See	See
42	KNeighborsClassifier_brute_force	predict	100000	1	2	0.239	0.004	0.0	0.239	1	5	1.000	1.936	0.003	0.124	0.002	See	See
43	KNeighborsClassifier_brute_force	predict	100000	1	2	0.239	0.004	0.0	0.239	1	5	1.000	0.003	0.000	86.156	6.644	See	See
44	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.712	0.009	0.0	0.004	1	1	0.896	1.070	0.005	3.468	0.019	See	See
45	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.712	0.009	0.0	0.004	1	1	0.896	0.002	0.000	2217.579	279.482	See	See
46	KNeighborsClassifier_brute_force	predict	100000	1	2	0.238	0.004	0.0	0.238	1	1	1.000	1.070	0.005	0.222	0.004	See	See
47	KNeighborsClassifier_brute_force	predict	100000	1	2	0.238	0.004	0.0	0.238	1	1	1.000	0.002	0.000	142.110	18.064	See	See

KNeighborsClassifier_kd_tree

onnx (1.10.1) vs. scikit-learn (1.0.dev0)

All estimators share the following hyperparameters: algorithm=kd_tree.

	estimator	function	n_samples_train	n_samples	n_features	mean_duration	std_duration	iteration_throughput	latency	n_jobs	n_neighbors	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	111.325	0.000	0.0	0.111	-1	1	0.929	0.472	0.007	235.818	3.267	See	See
1	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	111.325	0.000	0.0	0.111	-1	1	0.929	0.003	0.000	43303.980	6633.361	See	See
2	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.558	0.061	0.0	2.558	-1	1	1.000	0.472	0.007	5.420	0.150	See	See
3	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.558	0.061	0.0	2.558	-1	1	1.000	0.003	0.000	995.213	154.286	See	See
4	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	108.213	0.000	0.0	0.108	-1	5	0.946	0.893	0.006	121.129	0.864	See	See
5	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	108.213	0.000	0.0	0.108	-1	5	0.946	0.003	0.001	34221.935	7473.963	See	See
6	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.582	0.093	0.0	2.582	-1	5	1.000	0.893	0.006	2.890	0.106	See	See
7	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.582	0.093	0.0	2.582	-1	5	1.000	0.003	0.001	816.460	180.733	See	See
8	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	110.673	0.000	0.0	0.111	1	100	0.951	5.156	0.010	21.465	0.044	See	See
9	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	110.673	0.000	0.0	0.111	1	100	0.951	0.003	0.001	34420.526	10234.645	See	See
10	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.584	0.074	0.0	2.584	1	100	1.000	5.156	0.010	0.501	0.014	See	See
11	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.584	0.074	0.0	2.584	1	100	1.000	0.003	0.001	803.522	240.014	See	See
12	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	121.503	0.000	0.0	0.122	-1	100	0.951	2.952	0.012	41.158	0.167	See	See
13	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	121.503	0.000	0.0	0.122	-1	100	0.951	0.005	0.001	23652.753	5655.479	See	See
14	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.590	0.094	0.0	2.590	-1	100	1.000	2.952	0.012	0.877	0.032	See	See
15	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.590	0.094	0.0	2.590	-1	100	1.000	0.005	0.001	504.213	121.953	See	See
16	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	121.827	0.000	0.0	0.122	1	5	0.946	1.530	0.005	79.636	0.268	See	See
17	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	121.827	0.000	0.0	0.122	1	5	0.946	0.002	0.000	75817.732	23457.794	See	See
18	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.586	0.075	0.0	2.586	1	5	1.000	1.530	0.005	1.690	0.049	See	See
19	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.586	0.075	0.0	2.586	1	5	1.000	0.002	0.000	1609.381	500.129	See	See
20	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	124.189	0.000	0.0	0.124	1	1	0.929	0.815	0.006	152.438	1.110	See	See
21	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	124.189	0.000	0.0	0.124	1	1	0.929	0.001	0.000	123753.989	35718.018	See	See
22	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.584	0.104	0.0	2.584	1	1	1.000	0.815	0.006	3.172	0.130	See	See
23	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.584	0.104	0.0	2.584	1	1	1.000	0.001	0.000	2575.015	750.369	See	See
24	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.038	0.001	0.0	0.000	-1	1	0.891	0.021	0.001	1.835	0.097	See	See
25	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.038	0.001	0.0	0.000	-1	1	0.891	0.002	0.000	17.104	2.851	See	See
26	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	1	1.000	0.021	0.001	0.249	0.017	See	See
27	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	1	1.000	0.002	0.000	2.324	0.402	See	See
28	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.039	0.001	0.0	0.000	-1	5	0.911	0.022	0.001	1.748	0.070	See	See
29	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.039	0.001	0.0	0.000	-1	5	0.911	0.002	0.000	16.330	3.099	See	See
30	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	5	1.000	0.022	0.001	0.233	0.012	See	See
31	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	5	1.000	0.002	0.000	2.176	0.418	See	See
32	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.060	0.000	0.0	0.000	1	100	0.894	0.032	0.000	1.869	0.015	See	See
33	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.060	0.000	0.0	0.000	1	100	0.894	0.001	0.000	99.105	27.825	See	See
34	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	100	1.000	0.032	0.000	0.161	0.009	See	See
35	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	100	1.000	0.001	0.000	8.544	2.448	See	See
36	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.061	0.002	0.0	0.000	-1	100	0.894	0.035	0.002	1.750	0.115	See	See
37	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.061	0.002	0.0	0.000	-1	100	0.894	0.002	0.000	29.760	6.528	See	See
38	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	100	1.000	0.035	0.002	0.150	0.011	See	See
39	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	100	1.000	0.002	0.000	2.554	0.565	See	See
40	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.038	0.001	0.0	0.000	1	5	0.911	0.020	0.000	1.940	0.041	See	See
41	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.038	0.001	0.0	0.000	1	5	0.911	0.001	0.000	65.631	17.946	See	See
42	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	5	1.000	0.020	0.000	0.262	0.014	See	See
43	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	5	1.000	0.001	0.000	8.878	2.466	See	See
44	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.038	0.000	0.0	0.000	1	1	0.891	0.018	0.000	2.062	0.041	See	See
45	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.038	0.000	0.0	0.000	1	1	0.891	0.001	0.000	63.898	16.870	See	See
46	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	1	1.000	0.018	0.000	0.280	0.011	See	See
47	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	1	1.000	0.001	0.000	8.680	2.311	See	See

HistGradientBoostingClassifier_best

onnx (1.10.1) vs. scikit-learn (1.0.dev0)

All estimators share the following hyperparameters: learning_rate=0.01, n_iter_no_change=10.0, max_leaf_nodes=100.0, max_bins=255.0, min_samples_leaf=100.0, max_iter=300.0.

	estimator	function	n_samples_train	n_samples	n_features	mean_duration	std_duration	n_iter	iteration_throughput	latency	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	HistGradientBoostingClassifier_best	predict	100000	1000	100	0.462	0.019	300	0.002	0.000	0.824	0.106	0.001	4.346	0.178	See	See
1	HistGradientBoostingClassifier_best	predict	100000	1000	100	0.462	0.019	300	0.002	0.000	0.824	0.017	0.001	27.855	1.520	See	See
2	HistGradientBoostingClassifier_best	predict	100000	1	100	0.383	0.010	300	0.000	0.383	1.000	0.106	0.001	3.599	0.099	See	See
3	HistGradientBoostingClassifier_best	predict	100000	1	100	0.383	0.010	300	0.000	0.383	1.000	0.017	0.001	23.072	1.046	See	See